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Abstract 



We consider a key exchange procedure whose security is based on the difficulty of computing 
Cdiscrete logarithms in a group, and where exponentiation is hidden by a conjugation. We give a 
^^latform- dependent cryptanalysis of this protocol. Finally, to take full advantage of this procedure, 
Ctye propose a group of matrices over a noncommutative ring as platform group. 

C/3 ! Keywords: Key exchange, Quasideterminant, Noncommutative determinant. 
1. Introduction 

■ The Diffie-Hellman key agreement protocol is the first published practical solution to the key 
Slistribution problem, allowing two parties that have never met to exchange a secret key over an open 
flannel. It uses the cyclic group F* where ¥ q is the finite field with q elements. The security of this 
'"protocol is based on the difficulty of computing discrete logarithms in the group F*. 
r-There are several algorithms for computing discrete logarithms, some of them are subexponential 
^hen applied to F*. 

^ It is important to search for easily implementable groups, for which the DL problem is hard and 
there is no subexponential time algorithm for computing DL. The group of points over ¥ q of an 
^lliptic curve is such a group. 

C9o keeping in mind the above remarks and the fact that F* = GLi(¥ q ), one can wonder whether 
the group GL 2 {¥ q ) of two-by-two invertible matrices or more generally the group GL n (¥ q ), which 
.^dmit a "natural" normal form, can be used for a Diffie-Hellman protocol and whether there is some 
Advantage in using them. 

S 

Remark 1.1 Let us fix a matrix X G GL n (¥ q ). Knowing X and a power X a , is it easy to find a? 
The first point is that knowing X, one can compute det(X) e F* (the determinant of X), and also 
det(X a ) = (det(X)) a . In this way, the DL problem in matrix groups reduces to the DL problem in F*. 

One can avoid this difficulty by choosing a matrix X such that det(X) = I, but then by comput- 
ing eigenvalues of X and of X a ( possibly in an extension of the base field), and using the fact that 
the latter are the former in the power of a, one reduces once again the DL problem to the one in 
some extension of F*. 

So there is no advantage of considering the DL problem in the group of invertible matrices over 
a finite field, and more generally over a finite commutative ring. 

We wish to mention that the group of matrices over a finite field as above was first proposed as 
a platform group for Diffie-Hellman key exchange in [12], and was cryptanalysed using eigenvalues 
and Jordan form in [10]. Note that in this proposition the noncommutative structure of GL n (¥ q ) is 
not used. 



In [2], a protocol using noncommutative (semi) groups in cryptography was proposed. A platform 
using braid groups and the same idea was proposed in [9]. Also another platform using matrix algebra 
was discussed in [16]. The protocol we use in section 3 is based on the same idea. It uses conjugation 
and exponentiation together for its security. A platform for this protocol using braid groups was first 
proposed in [14] and another one using an F g -algebra in [11]. We shall give a cryptanalysis of these 
two platforms in section 3, by reducing the problem to the discrete logarithm problem over some 
finite field . 

The semigroup of matrices over a commutative ring was considered in [8] for an authentication 
protocol, but its security is based on the difficulty of the conjugacy search problem and not on the 
discrete logarithm one. In fact the authors consider matrices over a somehow complicated ring, 
namely the ring of N-truncated polynomials in k variables to make the conjugacy search problem 
infeasible. 

To avoid the reduction of DL problem to the one over finite fields mentioned in the above remark, 
which stems from the special features of (semi)-group matrices over finite fields (namely determinant 
and properties of eigenvalues), we can consider matrices over noncommutative finite rings. Group 
algebras F ? [G], where G is a noncommutative finite group are examples of such rings. The simplest 
example of such group algebras is the group algebra of the group of permutations of three elements, 
which is easily implementable. We can then consider two-by-two invertible matrices over such a 
group algebra. In the next section, one considers matrix groups over noncommutative rings and 
investigate whether the previously mentioned reduction (remark 1.1) in the case of DL problem in 
the group matrices over finite fields can happen or not. 

2. Quasideterminants, noncommutative determinants, eigenvalues... 

Since the invention of quaternions, there has been attempts to define a notion of determinant of 
a matrix with noncommutative entries. Here one can mention great names such as Cayley, Study, 
Moore, Wedderburn, Heyting and Richardson, Ore, Dieudonne, Berezin, who all considered such 
noncommutative determinants. In most of the cases, these noncommutative determinants are ratio- 
nal functions of the entries. The most recent and most general attempt (1991) is due to I. Gelfand 
and Retakh. It proved to be very effective in many areas of noncommutative algebra. In what 
follows we recall some definitions and results from [4], [5], [6], [7]. See also [15], for a generalization 
of Dieudonne determinant. 

Given a square matrix A of size n, with entries in a noncommutative ring R, we note A^ the 
matrix obtained from A by deleting the ith row and the jth column. We also note by r\ the zth 
row of A with jth position excluded, and by the jth column of A with the zth position excluded. 
For each position the quasideterminant of A is defined by \A\ij := — (A JJ ) _1 c^. We have 

\A\ij e R and, of course, this quasideterminant exists if the (n — l)-by-(n — 1) matrix A 1 ^ is invertible. 
So, for a matrix of size n, there are n 2 quasideterminants. 



Remark 2.1: Even in the commutative case, a quasideterminant is equal not to a determinant, 
but to the ratio of two determinants, namely, \A\ij = (— l) t+J ■ 
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Using quasideterminants, one defines a noncommutative determinant which gives the determinant 
(modulo a sign) in the commutative case: 

Let / = {ii, %2, i n } and J = {ji, j 2 , ■■■,j n } De two orderings of the set {1, 2, 3, n}. Note 
by J 4*i*2---«fe,iu2---ife ^fie matrix obtained from A by deleting the lines ii,i2,---,ik an d the columns 
ji, j2, ■■■,jk- Then one defines the noncommutative determinant of the n-by-n matrix A by: 

f) ( A\ — I 4 1. . I 4*1,51 1 . . I Ahi2,jij2\. . I Ahi2-i„-i,ji,j 2 ...j n -i\. . 

Example: For a two-by-two matrix A = J 0,11 0,12 h we find I = J = {1,2} and Dj j = 

\ a 21 a 22 J 

(an — ai2<222 a 2l) a 22 — a ll°22 ~~ Ol2 a 22 a 21 a 22 

Using this noncommutative determinant one can recover some of the previously considered no- 
tions such as the Dieudonne determinant. 



There is still another definition of a noncommutative determinant [4], motivated by representa- 
tion theory, and giving the determinant in the commutative case. This noncommutative determinant 
is an elementary symmetric function of the noncommutative eigenvalues of A. We do not give this 
definition here. 



To summarize, there is an active area of noncommutative algebra dealing with noncommutative 
determinants, noncommutative eigenvalues, ... From our cryptographic point of view, we only need 
to make sure that there is no formula reducing the DL problem in the group of matrices with noncom- 
mutative entries to the DL problem in the ring of coefficients. To the best of our knowledge, there 
is no way to relate the determinant of a matrix or its eigenvalues to the corresponding determinant 
and eigenvalues of a power of this matrix in the noncommutative case. 

3. A Diffie-Hellman key exchange protocol 

We consider the following protocol, which is based on the general idea of [2]. The platform pro- 
posed in [9] using braid groups is based on the same idea; in the latter case, the security is based 
on the conjugacy search problem, whereas in the following, one uses the discrete logarithm and the 
conjugacy search problem together. 

Suppose G is a noncommutative group and Hi and H 2 two subgroups of G such that every element 
of Hi commutes with every element of H2. 

Here G, Hi, H 2 and an element iGGof some high order n will be public data. Alice and Bob will 
use these data to exchange a key. 

Alice selects at random a secret integer a G {2,3, ...,n — 1} and a secret element T G Hi 
(TX ^ XT); she computes TX a T~ x and sends it to Bob. 

Bob selects at random a secret integer b G {2,3,..., n — 1} and a secret element T' G H 2 
(T'X ^ XT'); he computes T'A^T'- 1 and sends it to Alice. 

Alice computes (T 1 X b T'~ r ) a = T 1 X ah T'~ l ; then she conjugates it by her secret element T to 
obtain TT' X ab T'~ l T~ l . 

Bob computes (TX a T^ 1 ) b = TX ab T^ and he conjugates it by his secret element T' to obtain 
T'T x ah T~ x T'~ x which is the same as what Alice obtained due to the commutativity TT' = T'T. 

We immediately see that the choice of a matrix group over a finite field (and to some extent over 
a commutative ring) as a platform group for this protocol is not a good one. In fact, Remark 1.1 in 
the introduction about the reduction of the DL problem from matrix groups to the same problem 
over some extension of the base field remains valid. Let A be an eigenvalue of TX a T~ x . One has 



det(TX a T- 1 - Xid) = 0, so det(T(X a - XidjT' 1 ) = 0. Then, by the multiplicative property of 
determinant, we get det(X a — Xid) = and A is an eigenvalue of X a and is equal to some eigenvalue 
of X to the power a. 

So choosing a matrix group over a finite field as a platform group offers no advantage. Furthermore, 
taking the underlying multiplicative group of an algebra as platform group does not provide any 
advantage either, as using representation theory one can reduce the problem to the one over matrices 
and then to the discrete logarithm over some finite field. 

This protocol was first used in [14] in the context of braid groups. In the paper the authors 
consider a modified irreducible Burau type representation of a braid group and apply this protocol 
at the representation level to the matrices over some finite field. By what we said previously this is 
not a good choice and can be reduced to the DL problem over some extension of the field. 
The same protocol was used in [11], by taking as the platform group the multiplicative group of a 
noncommutative algebra of dimension four over a finite field. By taking the regular representation of 
this algebra we can transfer the scheme to the level of matrices and then reduce it to the DL problem 
in some extension of the finite field. 

In [13] this protocol is implemented as a software for smartphones using (5 x 5) matrix groups over 
a finite field, and its performance is compared to other implementations using finite fields or elliptic 
curves. The result of this comparison is that this protocol is largely more performant than those 
using finite fields or elliptic curves. As mentioned before, due to the reduction to the case of discrete 
logarithm over a finite field, the performance of this protocol using matrix groups over a finite field 
must not be so different from the one over a finite field. 

So, to take the best advantage of this protocol, we propose to choose as a platform group the group 
of matrices over a noncommutative rings, namely we consider two by two matrices over the group 
algebra of the symetric group S 3 , which we denote by G = GL 2 (¥ q [S 3 ]) where S3 is the group of 
permutations of three elements. Here X will be an element of GL 2 (F ? [5 , 3]) and we fix 

H = H 1 = H 2 = {(1 l^je GL 2 (¥ q ) I x G ¥ q ,y G ¥ g ,x 2 - y V o}, 

which is a commutative subgroup of GL 2 (¥ q [S 3 ]). In fact H is a maximal torus of GL 2 {¥ q ). 
3.1 ElGamal encryption 

Suppose that Alice is the owner of the public key data, GL 2 (¥ q [G}), X G GL 2 (¥ q [G\) of order n 
and H = Hi = H 2 as above. Suppose also that Alice has selected a secret integer a and a secret 
matrix T E H, and made TX a T^ public. Bob can encrypt a message M intended for Alice, as 
follows: 

Bob selects a random integer b G {2, 3, .., n — 2}, and a matrix T' G H; 
he computes TT'X^T^T'^ 1 as explained in the precedent section. 

Bob determines a symmetric encryption key t based on TV 'x^T^T'' 1 (in a way he agreed upon 
with Alice). 

Bob uses an agreed upon symmetric encryption method with key t to encrypt M, resulting in the 
encryption E. 

Bob sends {T'X b T'-\E) to Alice. 

Receiving these data, Alice computes TT'X^T^T'- 1 , as in the previous section; she derives from 
this the symmetric encryption key t; she uses the agreed upon symmetric encryption method with 
key t to decrypt E, and finds M. 

Remark 3.1.1 The ElGamal encryption as explained above is an hybrid version of ElGamal's 
encryption. In the textbook ElGamal encryption, we can take the message M G GL 2 (¥ q [G]): 
Bob sends to Alice {T'X b T'- 1 ,TT'X ab T- l T'~ l M). 



Alice computes (TT'X ab T *T' x ) 1 and by multiplying at the left with the second data, she finds M. 
See also [1]. 

4. Choice of parameters and security 

Owing to the similarity between the protocol we use and the one proposed in the context of braid 
groups [9], one may ask if the same kind of attacks as in the braid groups can be applied in our context. 

We remind that the security of braid-based cryptography relies on the difficulty of the conjugacy 
search problem. The problem is as follows: Knowing an element X and a conjugate TXT -1 , is it 
easy to find T? In other words, we know an element and some conjugate of it and one tries to find a 
conjugating element T. One of the main attacks against these procedures is to search T not in the 
whole conjugacy class of X, but in some characteristic part of it. The second kind of attack is to use 
some probabilistic research in the conjugacy class of X. The third one is to use linear representations 
of braid groups to reduce the problem to the one in a matrix group, which is easy to solve. See [3] 
for details. 

The main difference between our approach and those using braid groups is that, in our case, X 
is publicly known, but the conjugacy class which is involved is that of X a , which is not known, so 
all the above attacks are useless in our case. 

As we mentioned before (section 2), specific features of the group of invertible matrices with 
noncommutative entries cannot be used to attack our protocol. 

As for the existing algorithms computing discret logarithms, such as "Baby Step, Giant Step", or 
the Pollard rho algorithm, they cannot be applied directly and without modification to our protocol, 
because in these algorithms one is supposed to know an element and some power of it; in our case 
X E G is known but X a is hidden due to the conjugation by a secret matrix T. 

Algorithm 4.1 We propose the following algorithm (an adaptation of the Baby Step Giant Step 
algorithm) for computing the secret keys. Let n be the order of X. So knowing X and Y = TX a T~ x , 
we want to compute the secret keys a , T and the exchanged key TT < x ah T'~ l T~ l . 

1) For k — 1 to n compute X k , and put the sorted result in a table. 

2) For x,y e¥ q such that x 2 — y 2 ^ put T x>y = ( X ^ J ; then compute T Xty YT~y, and compare 

\ y x J 

it to the table of step (1). 

3) If, for some k and some T XO:Vo , one has T XoM YT~^ = X k °, then stop step (2); a = k and 
T = being known, compute (T" X h T'~ l ) a and conjugate it by to obtain the exchanged 
secret key. 

As for the complexity of Algorithm 4.1, we have 0(n) group operations in the first step. Then, 
in the second step, we have 0(q 2 ) group operations and 0{nln{n)) comparaisons. So, assuming that 
a comparaison is much faster than a group operation, we conclude that altogether the algorithmic 
cost is 0(max(n,q 2 )). 

Taking into account the above values, we propose to take |F g | ~ 2 40 and the matrix X of GL 2 (¥ g [S 3 ] 
to be of order > 2 80 . 

We propose to generate the invertible matrices X as follows. First, we observe that every 
matrix ( ^ ? ) with a and b invertible in FJS3] and no condition on c is invertible, with in- 



verse ( a a ^_^ Y Also every matrix ( a ^ ) satisfying the same conditions is invert - 



ible, with inverse ^ b^ca^ 1 b^ 1 ) ' Then, we can see that ever Y matrix of the form X = 

( U , , ^ _i, i with u invertible in F-JSV1) and no condition on b,c is invertible as well. In- 
y c 1 + cu b J " " 



deed, we observe that PXQ = Id where P and Q are the invertible matrices P = 



1 

-cu^ 1 1 



and Q = (^ U q \ ^ ^ , leading to X 1 = ^ U ^_^° U ^ ^ . By multiplying invert- 
ible matrices of the types above, one can obtain a number of invertible matrices. 

We now determine |GL 2 (F g [S'3])|, which is helpful for computing the order of elements. 

Lemma 4.1.1: Suppose the characteristic of ¥ q is not 2 or 3, so that FjSy is a semisimple 
algebra. Then \GL 2 (¥ q [S 3 })\ = q 8 (q - l) 8 (g + 1)V + l)(g 2 + q + 1). 

Proof. Using the linear representations of the symmetric group S3 and of the group alge- 
bra F 9 [S3], namely the fact that S3 has three irreducible representations, two of dimension one 
and the third of dimension two, one can write FjSy ~ ¥ q © ¥ q © Mat2{¥ q ) (Wedderburn theo- 
rem). Then we find Mat 2 (¥ q [S 3 ]) ~ Mat 2 i¥ q ) © Mat 2 (¥ q ) © Mat 2 (Mat 2 (¥ q )), and GL 2 (¥ q [S 3 ]) ~ 
GL 2 ((¥ q )@GL 2 (¥ q )@GL A (¥ q ), whence \GL 2 (¥ q [S 3 ])\ = [ q ( q -lY(q+l)}%*-l)( q *- q ) q *- q *)( q *- q Z), 
and \GL 2 (¥ q [S 3 })\ = q\q - lf{q + 1) V + l){q 2 + q + 1). 

5. Conclusion 

Matrix groups admit a natural normal form, making them easy to use for cryptography. Over 
finite fields special properties of matrix groups such as determinant and eigenvalues can be used to 
develop attacks against the protocol investigated in this paper. So, in any cryptographic protocol 
using matrix groups, one has first to verify that the above properties cannot be used to defeat the 
system. By using matrix groups over a noncommutative ring such as the group algebra of a finite 
group (for instance Fj^]), we can avoid such attacks. 



We thank the referee for informing us of some references. 
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